Goto

Collaborating Authors

 design dimension




Self-Supervised Learning on Molecular Graphs: A Systematic Investigation of Masking Design

Yang, Jiannan, Thost, Veronika, Ma, Tengfei

arXiv.org Artificial Intelligence

Self-supervised learning (SSL) plays a central role in molecular representation learning. Yet, many recent innovations in masking-based pretraining are introduced as heuristics and lack principled evaluation, obscuring which design choices are genuinely effective. This work cast the entire pretrain-finetune workflow into a unified probabilistic framework, enabling a transparent comparison and deeper understanding of masking strategies. Building on this formalism, we conduct a controlled study of three core design dimensions: masking distribution, prediction target, and encoder architecture, under rigorously controlled settings. We further employ information-theoretic measures to assess the informativeness of pretraining signals and connect them to empirically benchmarked downstream performance. Our findings reveal a surprising insight: sophisticated masking distributions offer no consistent benefit over uniform sampling for common node-level prediction tasks. Instead, the choice of prediction target and its synergy with the encoder architecture are far more critical. Specifically, shifting to semantically richer targets yields substantial downstream improvements, particularly when paired with expressive Graph Transformer encoders. These insights offer practical guidance for developing more effective SSL methods for molecular graphs.


Checklist 1. For all authors (a)

Neural Information Processing Systems

Do the main claims made in the abstract and introduction accurately reflect the paper's Did you discuss any potential negative societal impacts of your work? Have you read the ethics review guidelines and ensured that your paper conforms to them? Did you state the full set of assumptions of all theoretical results? We mainly focus on empirical study in this work. If you ran experiments... (a) Did you specify all the training details (e.g., data splits, hyperparameters, how they were Did you report error bars (e.g., with respect to the random seed after running experiments Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?



approach to study GNN designs, the first quantitative analysis for GNN task similarity, and offers rigorous findings via 2

Neural Information Processing Systems

We thank the reviewers for their constructive feedback. We thank R2 and R3 for raising that our paper lacks theoretical analysis. LU activation significantly improves GNN performance. We will add these new discussions to the revised paper. We thank reviewers for suggesting other design dimensions to explore.



TSGym: Design Choices for Deep Multivariate Time-Series Forecasting

Liang, Shuang, Hou, Chaochuan, Yao, Xu, Wang, Shiping, Jiang, Minqi, Han, Songqiao, Huang, Hailiang

arXiv.org Artificial Intelligence

Recently, deep learning has driven significant advancements in multivariate time series forecasting (MTSF) tasks. However, much of the current research in MTSF tends to evaluate models from a holistic perspective, which obscures the individual contributions and leaves critical issues unaddressed. Adhering to the current modeling paradigms, this work bridges these gaps by systematically decomposing deep MTSF methods into their core, fine-grained components like series-patching tokenization, channel-independent strategy, attention modules, or even Large Language Models and Time-series Foundation Models. Through extensive experiments and component-level analysis, our work offers more profound insights than previous benchmarks that typically discuss models as a whole. Furthermore, we propose a novel automated solution called TSGym for MTSF tasks. Unlike traditional hyperparameter tuning, neural architecture searching or fixed model selection, TSGym performs fine-grained component selection and automated model construction, which enables the creation of more effective solutions tailored to diverse time series data, therefore enhancing model transferability across different data sources and robustness against distribution shifts. Extensive experiments indicate that TSGym significantly outperforms existing state-of-the-art MTSF and AutoML methods. All code is publicly available on https://github.com/SUFE-AILAB/TSGym.